Recovering Dense Tissue Multispectral Signal from in vivo RGB Images

Hyperspectral/multispectral imaging (HSI/MSI) contains rich information clinical applications, such as 1) narrow band imaging for vascular visualisation; 2) oxygen saturation for intraoperative perfusion monitoring and clinical decision making [1]; 3) tissue classification and identification of pathology [2]. The current systems which provide pixel-level HSI/MSI signal can be generally divided into two types: spatial scanning and spectral scanning. However, the trade-off between spatial/spectral resolution, the acquisition time, and the hardware complexity hampers implementation in real-world applications, especially intra-operatively. Acquiring high resolution images in real-time is important for HSI/MSI in intra-operative imaging, to alleviate the side effect caused by breathing, heartbeat, and other sources of motion. Therefore, we developed an algorithm to recover a pixel-level MSI stack using only the captured snapshot RGB images from a normal camera. We refer to this technique as "super-spectral-resolution". The proposed method enables recovery of pixel-level-dense MSI signals with 24 spectral bands at ~11 frames per second (FPS) on a GPU. Multispectral data captured from porcine bowel and sheep/rabbit uteri in vivo has been used for training, and the algorithm has been validated using unseen in vivo animal experiments.Comment: accepted by Hamlyn Symposium 201

Detecting the Sensing Area of A Laparoscopic Probe in Minimally Invasive Cancer Surgery

In surgical oncology, it is challenging for surgeons to identify lymph nodes and completely resect cancer even with pre-operative imaging systems like PET and CT, because of the lack of reliable intraoperative visualization tools. Endoscopic radio-guided cancer detection and resection has recently been evaluated whereby a novel tethered laparoscopic gamma detector is used to localize a preoperatively injected radiotracer. This can both enhance the endoscopic imaging and complement preoperative nuclear imaging data. However, gamma activity visualization is challenging to present to the operator because the probe is non-imaging and it does not visibly indicate the activity origination on the tissue surface. Initial failed attempts used segmentation or geometric methods, but led to the discovery that it could be resolved by leveraging high-dimensional image features and probe position information. To demonstrate the effectiveness of this solution, we designed and implemented a simple regression network that successfully addressed the problem. To further validate the proposed solution, we acquired and publicly released two datasets captured using a custom-designed, portable stereo laparoscope system. Through intensive experimentation, we demonstrated that our method can successfully and effectively detect the sensing area, establishing a new performance benchmark. Code and data are available at https://github.com/br0202/Sensing_area_detection.gitComment: Accepted by MICCAI 202

Fluorescence lifetime imaging microscopy: in vivo application to diagnosis of oral carcinoma

A compact clinically compatible fluorescence lifetime imaging microscopy (FLIM) system was designed and built for intraoperative disease diagnosis and validated in vivo in a hamster oral carcinogenesis model. This apparatus allows for the remote image collection via a flexible imaging probe consisting of a gradient index objective lens and a fiber bundle. Tissue autofluorescence (337 nm excitation) was imaged using an intensified CCD with a gate width down to 0.2 ns. We demonstrate a significant contrast in fluorescence lifetime between tumor (1.77±0.26 ns) and normal (2.50±0.36 ns) tissues at 450 nm and an over 80% intensity decrease at 390 nm emission in tumor versus normal areas. The time-resolved images were minimally affected by tissue morphology, endogenous absorbers, and illumination. These results demonstrate the potential of FLIM as an intraoperative diagnostic technique

Surgical polarimetric endoscopy for the detection of laryngeal cancer

The standard-of-care for the detection of laryngeal pathologies involves distinguishing suspicious lesions from surrounding healthy tissue via contrasts in colour and texture captured by white-light endoscopy. However, the technique is insufficiently sensitive and thus leads to unsatisfactory rates of false negatives. Here we show that laryngeal lesions can be better detected in real time by taking advantage of differences in the light-polarization properties of cancer and healthy tissues. By measuring differences in polarized-light retardance and depolarization, the technique, which we named 'surgical polarimetric endoscopy' (SPE), generates about one-order-of-magnitude greater contrast than white-light endoscopy, and hence allows for the better discrimination of cancerous lesions, as we show with patients diagnosed with squamous cell carcinoma. Polarimetric imaging of excised and stained slices of laryngeal tissue indicated that changes in the retardance of polarized light can be largely attributed to architectural features of the tissue. We also assessed SPE to aid routine transoral laser surgery for the removal of a cancerous lesion, indicating that SPE can complement white-light endoscopy for the detection of laryngeal cancer

Multispectral imaging of organ viability during uterine transplantation surgery in rabbits and sheep

Uterine transplantation surgery (UTx) has been proposed as a treatment for permanent absolute uterine factor infertility (AUFI) in the case of the congenital absence or surgical removal of the uterus. Successful surgical attachment of the organ and its associated vasculature is essential for the organ’s reperfusion and long-term viability. Spectral imaging techniques have demonstrated the potential for the measurement of hemodynamics in medical applications. These involve the measurement of reflectance spectra by acquiring images of the tissue in different wavebands. Measures of tissue constituents at each pixel can then be extracted from these spectra through modeling of the light–tissue interaction. A multispectral imaging (MSI) laparoscope was used in sheep and rabbit UTx models to study short- and long-term changes in oxygen saturation following surgery. The whole organ was imaged in the donor and recipient animals in parallel with point measurements from a pulse oximeter. Imaging results confirmed the re-establishment of adequate perfusion in the transplanted organ after surgery. Cornual oxygenation trends measured with MSI are consistent with pulse oximeter readings, showing decreased StO2 immediately after anastomosis of the blood vessels. Long-term results show recovery of StO2 to preoperative levels

Detecting the Sensing Area of a Laparoscopic Probe in Minimally Invasive Cancer Surgery

In surgical oncology, it is challenging for surgeons to identify lymph nodes and completely resect cancer even with pre-operative imaging systems like PET and CT, because of the lack of reliable intraoperative visualization tools. Endoscopic radio-guided cancer detection and resection has recently been evaluated whereby a novel tethered laparoscopic gamma detector is used to localize a preoperatively injected radiotracer. This can both enhance the endoscopic imaging and complement preoperative nuclear imaging data. However, gamma activity visualization is challenging to present to the operator because the probe is non-imaging and it does not visibly indicate the activity origination on the tissue surface. Initial failed attempts used segmentation or geometric methods, but led to the discovery that it could be resolved by leveraging high-dimensional image features and probe position information. To demonstrate the effectiveness of this solution, we designed and implemented a simple regression network that successfully addressed the problem. To further validate the proposed solution, we acquired and publicly released two datasets captured using a custom-designed, portable stereo laparoscope system. Through intensive experimentation, we demonstrated that our method can successfully and effectively detect the sensing area, establishing a new performance benchmark. Code and data are available at https://github.com/br0202/Sensing_area_detection.git